Cost Analysis of Games, Using Program Logic

Recent work in programming semantics has provided a relatively simple probablistic extension to predicate transformers, making it possible to treat small imperative probabilistic programs containing both demonic and angelic nondeterminism [11, 12, 20]. That work in turn was extended to provide a probabilistic basis for the modal μ-calculus [13], and leads to a quantitative μ-calculus [16, 18]. Standard (non-probabilistic) μ-calculus can be interpreted either ‘normally’, over its semantic domain, or as a two-player game between an ‘angel’ and a ‘demon’ representing the two forms of choice. It has been argued [23] that the two interpretations correspond. Quantitative μ-calculus can be interpreted both ways as well, with the novel interpretation being the second one: a probabilistic game involving an angel and a demon. Each player seeks a strategy to maximise (resp. minimise) the game’s ‘outcome’, with the steps in the game now being stochastic. That suggests a connection with Markov decision processes, in which players compete for high (resp. low) ‘rewards’ over a Markov transition system. In this paper we explore ‘the Markov connection’, showing for example how discounted Markov decision processes (MDP’s) and terminating MDP’s can be written as quantitative μ-formulae. The ‘normal’ interpretation of those formulae (i.e. over the semantic domain) then ∗Presented at the 8th Asia-Pacific Software Engineering Conference (APSEC 2001), 4–7 December 2001, University of Macau, Macau SAR, China. †Dept. of Computer Science and Engineering, University of New South Wales, 2052 Australia: [email protected] ‡Department of Computer Science and Mathematics, Macquarie University, 2109 Australia: [email protected]